This invention relates to digital image processing, and more particularly, to pattern and object recognition in a digital image.
Digital image processing systems that recognize patterns in images and detect objects within an image are well known. These systems use various conventional processing techniques for both of these tasks. For example, some of these systems are strictly mathematically based and use such algorithms as morphological processing, wavelet algorithms, convolutions, and the like while other systems use variations on neural net and fuzzy logic technologies. These systems have been used in a variety of image analysis environments and applications such as medical images obtained with ultrasound, x-ray, CAT scan, MRI and other known medical imaging systems to examine body tissue for the detection of abnormalities such as cancer. These systems have also been used in military image analysis of images generated by radar, sonar, infrared and other imaging systems to extract features such as terrain, infrastructure and the like and to detect specific targets such as tanks and other military vehicles, weaponry, camouflaged bunkers and the like. These systems have also been used in various commercial and scientific applications.
These previously known image analysis systems are extremely limited in the image environment in which they properly function. Additionally, the algorithms used by these systems to process images are brittle. That is, they are typically rendered ineffective by slight variations in image quality, image resolution, image noise level and many other factors. Consequently, these systems cannot be used for general pattern recognition and object detection across a wide range of applications, environments, or imaging modalities.
One of the most difficult image analysis environments is the ultrasound imaging of in vivo body tissues. The images of in vivo tissue generated by ultrasound equipment contain a high number of line variations and edge patterns, the detection of which is made difficult by the signal noise present in that imaging environment. Thus, image analysis based on a comparison of images of the same area of interest taken at different times, as typically done in systems analyzing ultrasound images, is inherently difficult.
With systems that use images to detect the presence of cancer, limitations arise from the knowledge base required by such systems and the computational resources available for image comparison. For example, neural networks may be used for image analysis systems; however, these systems require extensive training through the presentation of numerous images to the neural network. Because the images are presented to the system without intelligent insight as to the content of the images, the system tends to only learn statistically significant events and ignores image events less frequently encountered. For example, the submission of images of tissue having a particular abnormal cell structure indicative of some cancer may be learned by a neural network system to identify that structure and similar structures in tissue. However, a less statistically significant variant of that structure that may also indicate cancer would not be incorporated in the knowledge base of the system. Consequently, the system may not detect these variant structures which are also indicative of cancer. Another problem with previously known image analysis systems is the amount of resources required for image processing. The elements for processing images on a pixel by pixel basis can be significant as the number of pixels contained within an image increase. In some cases, the storage of multiple images so they may be later compared to one another can be prohibitive or require archival systems that increase the time required for image retrieval and processing.
What is needed is an image processing system and method that recognizes patterns and detects objects in images without requiring adaptation of the system to a particular application, environment, or image content.
What is needed is an image analysis system and method that maintains consistent image evaluation which is independent of variation in image modality, resolution, noise level, quality, and other image factors.
What is needed is an image analysis system and method for cancer detection that accurately analyzes in vivo images in a wide variety of applications.
What is needed is an image analysis system and method that simplifies the processing of images from the pixel by pixel comparisons performed by known systems.
What is needed is an image analysis system and method that incorporates within a knowledge base variant structures of an object or pattern that are not statistically significant.
What is needed is an image analysis system and method for feature extraction and target recognition in various military, commercial and scientific digital images.
The above limitations of previously known image processing systems are overcome by a system and method operating in accordance with the principles of the present invention. An image processing system made in accordance with the principles of the present invention includes a knowledge base of themes with the themes being coupled to knowledge elements by associative links, a synaptic link generator for generating synaptic links for pixels in a digital image, the synaptic link being used to identify a knowledge element for a corresponding pixel and a theme identifier for evaluating associative links for a plurality of knowledge elements corresponding to a theme to determine whether the corresponding theme is present in a digital image. This system may be used to analyze a digital image for patterns and objects in many different applications, environments, and imaging modalities.
In the system of the present invention, the synaptic link generator uses a data mask to select pixels from an image and generates a synaptic link from determinants generated for each pixel. Preferably, these determinants are generated from data for pixels lying in radials extending from the pixel under investigation. A plurality of determinants are preferably used to generate a synaptic link that identifies pattern information surrounding the pixel under investigation. These determinants encode pattern data in the vicinity of the target pixel so it may be more easily processed and recognized. A trainer then determines the identity of a knowledge element or knixel that corresponds to each pixel. This operation may be performed by having the trainer identify a window around an object within an image and identifying a knowledge element for that object so that each synaptic link generated for the pixels within the window is associated with the identified knowledge element. For example, an outline of a blood vessel may be defined within a digital image by a trainer and each synaptic link for each pixel within that window is associated with the knowledge element identified as a xe2x80x9cblood vessel.xe2x80x9d This type of learning is called area learning.
A target learning mode may also be used. In this learning mode, the synaptic links within a window are mapped to a knowledge element, then all portions of the image which the trainer recognizes as corresponding to the knowledge element being learned by the system are blocked out and the synaptic links for the remaining pixels of the image outside the active window and blocked out regions are set to a knowledge element identifier that indicates the knowledge element is unknown. This process breaks the synaptic links generated for xe2x80x9cnoisexe2x80x9d pixels within the active knowledge element window to the knowledge element identifier being learned. By repopulating the blocked out areas with image data and processing them, a set of synaptic links for the knowledge element being learned is developed that identify the objects corresponding to that knowledge element from different perspectives.
During a learning phase, a synaptic link may be generated for association with a knowledge element that differs from the knowledge element associated with the same synaptic link generated for another pixel. The system, in this case, notifies the trainer of this discrepancy and the trainer may resolve the discrepancy to identify a single knowledge element for the synaptic link. The process of identifying a single knowledge element for a synaptic link generated from pixel data within the image forms a synaptic array knowledge base for the system.
The relationship between synaptic links and a knowledge element is many to one. That is, more than one synaptic link value may be mapped to a knowledge element identifier. The knowledge element identifier is used to select an action that may modify the target pixel, generate data for a user, access an external system, or link to another knowledge element or theme.
Also during the training period, thematic elements are likewise identified by a trainer. Typically, these thematic elements are defined by a region of interest (xe2x80x9cROIxe2x80x9d) window that includes pixels associated with more than one type of knowledge element. That is, a theme is an object that is composed of other objects or knowledge elements. For example, an ROI window for a thematic element may include pixels that are associated with knowledge elements identified as hard tissue (bone), soft tissue, and blood vessels for medical images or runway lights, concrete and landing markers for a runway. The data structure for a thematic element includes a list of the associative links to various knowledge elements or themes for the thematic element. If the conditions of the associative links are met, then an action for a thematic element is performed. Actions for thematic elements correspond to the actions for knowledge elements discussed above.
The image processor of the system described above may be operated in accordance with the principles of the present invention to generate the knowledge bases for the image processor. In the training mode, the synaptic link generator generates a synaptic link for each pixel in a plurality of pixels in a digital image and a knowledge element identifier is stored in a location of the synaptic array corresponding to the generated synaptic links. The thematic identifier associates a plurality of knowledge elements to a thematic element with associative links. The actions stored in the knowledge element knowledge base and the thematic element knowledge base are defined by the trainer and stored in the knowledge bases. Once the system has been used to generate the knowledge bases for the image processing system, images may then be processed for detection of objects that correspond with one of the thematic elements in the thematic element knowledge base.
Processing of an image by the inventive image processing system includes the steps of defining an ROI window that is used to select pixels from a digital image. The associative links for the thematic element are processed by determining whether the knowledge element associated with each associative link meets the clustering criteria for knowledge elements and the logical functions defined by the form field of the associative link. The clustering criteria are set by a system or operator and may be selectively adjusted. If a correspondence is found between the relationship for each knowledge element type meeting the clustering criteria within an ROI window and the logical conditions for the associative links in the data structure identified for a thematic element are satisfied, then a theme is located within the ROI window. The action for the thematic element may then be performed. These actions may identify or highlight the theme in an image or generate some text appropriate for the theme detected within the image.
These and other advantages and benefits of the present invention may be ascertained from the detailed description of the invention in the accompanying drawings.